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Exposure to respirable elemental carbon (REC), a component of diesel exhaust (DE), was as- 
sessed for an epidemiologic study investigating the association between DE and mortality, par- 
ticularly from lung cancer, among miners at eight mining facilities from the date of 
dieselization (1947-1967) through 1997. To provide insight into the quality of the estimates 
for use in the epidemiologic analyses, several approaches were taken to evaluate the exposure 
assessment process and the quality of the estimates. An analysis of variance was conducted to 
evaluate the variability of 1998-2001 REC measurements within and between exposure groups 
of underground jobs. Estimates for the surface exposure groups were evaluated to determine if 
the arithmetic means (AMs) of the REC measurements increased with increased proximity to, 
or use of, diesel-powered equipment, which was the basis on which the surface groups were 
formed. Estimates of carbon monoxide (CO) (another component of DE) air concentrations 
in 1976-1977, derived from models developed to predict estimated historical exposures, were 
compared to 1976-1977 CO measurement data that had not been used in the model develop- 
ment. Alternative sets of estimates were developed to investigate the robustness of various 
model assumptions. These estimates were based on prediction models using: (i) REC medians 
rather AMs, (ii) a different CO:REC proportionality than a 1:1 relation, and (iii) 5-year aver- 
ages of historical CO measurements rather than modeled historical CO measurements and 
DE-related determinants. The analysis of variance found that in three of the facilities, most 
of the between-group variability in the underground measurements was explained by the 
use of job titles. There was relatively little between-group variability in the other facilities. 
The estimated REC AMs for the surface exposure groups rose overall from 1 to 5 |Ag m -3 
as proximity to, and use of, diesel equipment increased. The alternative estimates overall were 
highly correlated (~0.9) with the primary set of estimates. The median of the relative differ- 
ences between the 1976-1977 CO measurement means and the 1976-1977 estimates for six 
facilities was 29%. Comparison of estimated CO air concentrations from the facility-specific 
prediction models with historical CO measurement data found an overall agreement similar to 
that observed in other epidemiologic studies. Other evaluations of components of the exposure 
assessment process found moderate to excellent agreement. Thus, the overall evidence suggests 
that the estimates were likely accurate representations of historical personal exposure levels to 
DE and are useful for epidemiologic analyses. 
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INTRODUCTION 

Investigators at the US National Cancer Institute 
(NCI) and the US National Institute for Occupational 
Safety and Health (NIOSH) conducted the Diesel 
Exhaust in Miners Study (DEMS) to examine causes 
of mortality, particularly from lung cancer, associ- 
ated with diesel exhaust (DE) among underground 
miners. A retrospective exposure assessment was 
conducted to provide estimates of average exposure 
levels, by mining facility, department, job title, and 
year to respirable elemental carbon (REC), a compo- 
nent of DE. 

This paper describes an evaluation of the exposure 
assessment process. The goal was to determine the 
accuracy of the estimates developed for the study 
and whether the estimates were useful for investigat- 
ing mortality risk associated with DE exposure. 



Several approaches were used in the evaluation 
because there was no gold standard and because the 
independent data used in the evaluations varied in 
type, quantity, and quality by facility. Some informa- 
tion on these evaluations was presented in other re- 
ports (Stewart et al. , 20 1 0; Vermeulen et al. , 20 10a,b). 

An overview of the exposure assessment steps is 
presented with an indication where an evaluation 
was made (Table 1). A description of each evaluation 
is then presented. Results are presented in order of 
the evaluation description in the Methods section. 

METHODS 

Overview of the exposure assessment methods 

The primary objective of the exposure assessment 
process was to develop annual quantitative estimates 



Table 1, Summary of the independent data used for the evaluations. 



Step in exposure assessment process Type of evaluation; data evaluated 



Comparison data 



Processing of work histories 



Selection of the AM as the 
exposure metric 



Underground exposure 
estimates — development of 
underground exposure groups 



Underground exposure 
estimates — assumptions 
used for modeling historical 
trends 



Underground exposure 
estimates — time trends 



Surface exposure estimates 



Percent agreement; locations, 
identified by long-term workers, 
where study subjects worked 
who had missing location in 
work histories 

Pearson correlation; cumulative 
exposure levels of underground 
study subjects from estimates 
based on AMs of 1998-2001 
DEMS REC personal measurements 
and the primary prediction models 

Relative difference and Pearson 
correlation; TWAs based on 
estimates of proportion of time 
jobs spent in major underground 
areas and 1998-2001 DEMS area 
measurements 

Analysis of variance; estimated 
between- and within-group 
variability of REC measurements 

Relative difference: REC estimates 
for underground jobs derived from 
the primary prediction models 
using a 1:1 CO:REC proportionality 

Pearson correlation: REC 
cumulative exposure levels of 
underground study subjects from 
estimates derived from the primary 
prediction models using a 1:1 CO:REC 
proportionality 

Relative difference: 1976-1977 CO 
face area estimates derived from the 
primary prediction models in six facilities 

Relative difference: 1994 REC 
personal estimates in Facility B 

Monotonic trends: REC personal 
estimates and % NDs by 
surface groups 



Facility records (n = 1640 locations) 



Cumulative exposure levels of 
underground study subjects from 
estimates based on medians of 
1998-2001 DEMS REC personal 
measurements and the primary 
prediction models (n = 8344 subjects) 

1998-2001 DEMS REC personal 
measurements (n = 97 jobs) 



1998-2001 DEMS REC personal 

measurements (Ul n = 97 jobs; 

U2 n — 39 groups; U3 n = 22 groups) 3 

REC underground estimates derived 

from Yanowitz et al. (2000) and 

US EPA (2002) findings of a differential 

change of DPM to CO over time 

(n =172 facility-years) 

Cumulative REC estimates derived 
from: (i) REC = CO 0 58 relation seen 
in the 1998-2001 DEMS REC and CO 
area measurements, (ii) 1976-2001 
5-year average CO area measurements. 
Both n = 8344 subjects 

1976-1977 CO short-term face area 
measurement data n = 6 means) 
(Sutton et al, 1979) 

1994 REC personal measurements 
(n = 2 jobs) (Stanevich et al, 1997) 

1998-2001 DEMS personal 
measurements (n = 24 exposure groups) 



TWA, time-weighted average; DPM, diesel particulate matter; CO, carbon monoxide; ND, non detectable measurement. 
"See footnote b of Table 3 for definition of Ul, U2, and U3. 
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of DE exposure levels, using REC, for all jobs at 
eight mining facilities from the first year of diesel 
use (1947-1967, depending on the facility) to 31 
December 1997 for use in the epidemiologic analy- 
ses. The procedure was 

• REC and carbon monoxide (CO) personal and 
area measurements were collected during air 
monitoring surveys in 1998-2001 at seven of 
the eight study facilities (the DEMS surveys) 
(Coble et al, 2010; Vermeulen et al, 2010b). 
(The eighth facility had closed permanently in 
1993.) These measurements were, essentially, 
the only REC measurements available for the 
study facilities. Work histories were abstracted 
from the mining facilities' personnel records. 
Long-term workers provided information on the 
small number of work locations that were miss- 
ing from the work history records. The accuracy 
of this information was assessed (See Jobs with 
missing location information). 

• Exposure groups were developed independently 
of REC personal or area measurements and sep- 
arately for underground and for surface jobs 
based on exposure determinants collected during 
interviews with long-term workers. These groups 
then were compared to the REC measurements. 
(See Underground Exposure Estimates, Develop- 
ment of Underground Exposure Groups; Surface 
Exposure Estimates). 

• The arithmetic means (AMs) of the REC meas- 
urements on the subset of jobs measured in the 
DEMS surveys were assigned as the 1998-2001 
reference estimates to all jobs within each expo- 
sure group. A sensitivity analysis compared, as 
the exposure metric, the AM with the median 
(See REC AMs as the Exposure Metric). 

• Historical CO air concentrations at the produc- 
tion face were used as a surrogate for DE expo- 
sures and estimated back to the first year of 
dieselization in the underground locations. The 
concentrations were modeled using primarily 
US Mine Safety and Health Administration 
(MSHA) Mine Information Data System (MI- 
DAS) historical area CO compliance data and 
DE-related determinants [e.g. engine horsepower 
adjusted for percentage of time used (ADJ HP) 
and total air flow rate exhausted from the under- 
ground operations in cubic feet per minute 
(CFM), i.e. ADJ HP/CFM] (1 HP = 0.746 kW; 
1 CFM = 1.7 m 3 h~'). The results of this model- 
ing are called herein the primary prediction mod- 
els. Sensitivity analyses were conducted on the 
assumptions used in the models. (See Under- 



ground Exposure Estimates, Assumptions used 
for Modeling Historical Trends). 

• The annual modeled CO air concentrations rela- 
tive to 1998-2001 CO personal measurements 
from the DEMS surveys were used to adjust 
the job-specific 1998-2001 REC reference esti- 
mates back to the start of dieselization to provide 
historical annual REC personal exposure esti- 
mates. The final estimates were compared to in- 
dependent historical measurement data. (See 
Underground Exposure Estimates, Time Trends). 

• REC personal exposure estimates for surface 
jobs were adjusted for temporal changes only 
when jobs changed exposure groups (see Stewart 
et al, 2010) (such as when diesel-powered 
equipment replaced gasoline-powered equip- 
ment). This approach was taken because of the 
very low exposure levels of workers on the sur- 
face compared to those underground (Coble 
et al, 2010) and because of the less specific in- 
formation available on surface diesel equipment. 

• The estimates for both underground and surface 
jobs were then combined with individual work 
histories to derive estimates of personal cumula- 
tive exposure and other exposure metrics. The es- 
timates derived from the primary prediction 
models were considered the primary estimates 
for the epidemiologic analysis. 

Details and other information on the exposure as- 
sessment process are available elsewhere (Coble 
et al, 2010; Stewart et al, 2010; Vermeulen et al., 
2010a,b;). 

Jobs with missing location information 

About 5% of the job entries from the personnel re- 
cords lacked information on whether the location of 
the job was on the surface or underground. It was 
critical to obtain this information because of the 
large differences in exposure levels between surface 
and underground workers (Coble et al., 2010). Work 
histories of individuals with one or more jobs with an 
unknown location lasting at least 2 years were re- 
viewed with long-term employees at the individuals' 
mining facility to determine the work location. After 
the site visits had been completed, the work histories 
for two facilities that had particularly complex work 
history records were re-abstracted to ensure high 
quality data. In addition, for these and the other min- 
ing facilities, seniority records and other information 
in the personnel records not originally abstracted 
were reviewed for location and used to replace the 
missing information. For this evaluation, we calcu- 
lated the percent of overall agreement for the 
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locations identified during the interviews and the lo- 
cations from the additional information in the per- 
sonnel records, overall and by facility, by surface 
and underground classification. 

REC AMs as the exposure metric 

The AM was used in the exposure assessment pro- 
cess because AMs are considered the best statistic 
for calculating cumulative exposure in evaluations 
of chronic disease (Seixas et ah, 1991). Accordingly, 
the 1998-2001 job-specific REC reference estimates 
were based on the AM of the DEMS REC full-shift 
personal measurements (Coble et al., 2010). To in- 
vestigate the robustness of the AM to high or low 
measurements, a set of alternate REC exposure esti- 
mates was developed for all underground jobs by re- 
placing the 1998-2001 REC AMs by the 1998-2001 
medians of the REC measurements and using the rel- 
ative time trends derived from the primary prediction 
models. Cumulative exposure levels from both sets 
of estimates were calculated for all underground 
study subjects. Pearson correlation coefficients (r P ) 
were calculated overall and by mining facility. 

Underground exposure estimates 

Development of underground exposure groups. Be- 
cause of limited job-specific REC measurement 
data, we initially developed three types of exposure 
groups for each facility's underground operation us- 
ing the following criteria in order of decreasing spec- 
ificity (homogeneity): Ul, in which each group, 
represented by a single standardized job title, com- 
prised all job entry titles from the personnel records 
that differed only in spelling or word order and that 
facility personnel had identified as having performed 
the same job tasks; U2, sets of groups, each compris- 
ing several Ul groups whose workers spent similar 
proportions of time (as indicated in the interviews) 
in four major areas of the underground operations 
(i.e. the production face, haulage and travel ways, 
shop and office area, and crusher area); and U3, sets 
of U2 groups whose members spent time in areas 
with similar CO air concentrations based on the his- 
torical MIDAS CO area measurements (Coble et al. , 
2010). All groups were developed without regard to 
REC measurements. [Two other types of groups 
(contributing 14% of the person-years) were devel- 
oped but could not be evaluated; see Stewart et al., 
2010]. Once the groups were developed, the REC 
full- shift personal measurements from the DEMS 
survey were assigned to each group. Each job title 
was then assigned and blinded to the REC measure- 
ment levels, to one of the three exposure groups, 



based on its most specific group having at least five 
REC measurements. The AM of the REC personal 
measurements assigned to each group became the 
reference 1998-2001 exposure estimate. 

As indicated above, the grouping of jobs to the U2 
and, indirectly, the U3 exposure groups was depen- 
dent on the proportion of time workers in the various 
jobs spent in four major underground areas that dif- 
fered substantially with regard to DE concentrations 
(Coble et al., 2010). This assignment of time and 
area to each job could be tested by using those same 
proportions of times and the REC full-shift area 
measurements collected in those areas during the 
DEMS surveys (Coble et al, 2010). First, the AMs 
of the REC area measurements taken in each of the 
four areas were calculated for each facility (Coble 
et al., 2010). For each job measured in the DEMS 
survey, each area AM was weighted by the respec- 
tive proportion of a work shift that workers in the 
job spent in each area and summed across all areas 
to develop an 8-h time- weighted average (TWA). 
Differences and relative differences (the difference 
divided by the observed mean) were calculated be- 
tween the observed AMs, i.e. of the full-shift per- 
sonal measurements, and the predicted TWAs. 
Pearson correlation coefficients were also calculated 
between the predicted and observed values of each 
job by facility and overall. 

To determine how much variability in the REC 
personal measurements was explained by the three 
underground exposure groups (i.e. U1-U3), variance 
component models were constructed for each facility 
to estimate the between- and within-group variabil- 
ity of the measurements. For Ul groups (i.e. the stan- 
dardized job titles), a random effects model was 
used. For U2 groups (time spent in the four under- 
ground areas) and U3 groups (time spent in areas 
with similar CO air concentrations), mixed effects 
models were constructed, where the U2 or U3 groups 
were introduced into the model as fixed effects and 
the jobs within the exposure group were treated as 
random effects. 

Assumptions used for modeling historical trends. To 
develop historical estimates for underground jobs, 
reference REC personal exposure estimates were 
adjusted using facility-specific CO time trend pre- 
diction models. The models used the assumptions 
that a change in CO levels resulted in an equal 
change in REC levels and that this relationship was 
constant over time up to the early 1990s, i.e. a 1:1 
proportionality (Vermeulen et ah, 2010a). (Adjust- 
ment to this proportionality to account for cleaner 
engines after 1990 was made in the model but is 
not applicable to this discussion; see Vermeulen 
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et al., 2010a.) Diesel engines underwent technologic 
changes before the 1990s, however, and these 
changes affected emissions of CO and diesel partic- 
ulate matter (DPM, of which the primary component 
is REC (Ramachandran and Watts, 2003; Noll et al, 
2007). The US Environmental Protection Agency 
(US EPA, 2002) modeled data generated by 
Yanowitz et al. (2000) on emissions of diesel engines 
manufactured between 1976 and the late 1990s. The 
US EPA found that CO and DPM emissions in- 
creased with the year of manufacture at only 
a slightly different rate (annual parameter estimate 
difference = —0.003, i.e. that DPM emissions in- 
creased slightly less than CO emissions for each ear- 
lier manufacturing year). To estimate the effect of 
this finding on the primary model estimates, we cal- 
culated the mean purchase year of all diesel equip- 
ment in use underground using for each piece, the 
purchase year weighted by its ADJ HP. This proce- 
dure was done for each year for each facility from 
1997 back to 1976. The effect of the two sets of as- 
sumptions for each facility was calculated by com- 
paring the 1976 REC estimates derived from the 
primary models (i.e. with the 1:1 proportionality) 
and the 1976 REC estimates adjusted for the param- 
eter estimate of —0.003. The difference is the ap- 
proximate likely difference in our estimates as 
a result of changes in engine technology not already 
accounted for in the primary estimates. 

In addition, a sensitivity analysis was conducted 
based on the relation of REC = CO 0 58 seen in the 
DEMS area measurements (Vermeulen et al., 
2010a,b). An alternate set of historical prediction 
trends for underground jobs was developed for each 
facility using this relation, rather than the 1 : 1 propor- 
tionality used in the primary models. This set of 
models was called the REC = CO 0 ' 58 models. A sec- 
ond sensitivity analysis evaluated a set of time trend 
estimates developed from the observed 5-year aver- 
ages of the MSHA MIDAS historical CO measure- 
ments used for the prediction models, without the 
DE-related determinants. For years prior to 1976, 
when there were no CO measurements, the ratio of 
ADJ HP/CFM to the 1976 ADJ HP/CFM was used 
to adjust the CO averages (Vermeulen et al., 
2010a). These models were called the 5-year average 
CO models. A set of cumulative estimates for under- 
ground workers was calculated using the REC = 
CO 0 ' 58 models, as was a second set using the 5-year 
average CO models. Pearson correlations were cal- 
culated between the cumulative REC exposure levels 
derived from the primary estimates and the cumula- 
tive exposure levels derived from the two alternate 
sets of estimates. 



Time trends. Measurement data from an US Mine 
Enforcement and Safety Administration (MESA)/ 
US Bureau of Mines (BoM) monitoring study con- 
ducted in 1976-1977 in six of the study facilities 
(Sutton et al, 1979) were not used in the develop- 
ment of the primary prediction models and so could 
be used for method evaluation (Vermeulen et al., 
2010a). CO area measurements at the face were se- 
lected for comparison because the primary models 
estimated CO face concentrations and CO personal 
measurements were not available. Thus, the facility- 
specific AMs of the 1976-1977 face measurements 
were compared to the 1976-1977 CO estimates de- 
rived from the facility-specific primary models. A 
second comparison was made of the AMs of the 
REC full-shift personal measurements taken in 
1994 at one mining facility (B) (Stanevich et al., 
1997) to the 1994 REC estimates derived from the 
primary model for two of four underground jobs 
for which the data could be matched. For both the 
CO and REC comparisons, differences and relative 
differences were calculated between the predicted 
and observed values. 

Surface exposure estimates 

Interviews with long-term workers provided infor- 
mation on the frequency of use of, and proximity to, 
diesel-powered equipment. The information was 
used to categorize jobs into one of three exposure 
groups: (i) jobs in which workers had no or very 
limited contact with diesel equipment (Exposure 
group A), (ii) jobs in which workers drove a diesel 
forklift truck indoors or operated heavy equipment 
(>75 HP) <4 h per shift on average, drove light 
diesel equipment (<75 HP), or worked in close 
proximity to diesel-powered equipment on a regular 
basis (Exposure group B), and (iii) jobs in which 
workers operated heavy diesel equipment or drove 
a diesel forklift truck indoors for >4 h per shift on 
average, and mechanics and maintenance workers 
who repaired diesel equipment (Exposure group 
C). The DEMS REC personal measurements were 
assigned to each of the A, B, and C groups within 
each facility (SI estimation groups); across all 
facilities of the same ore type (i.e. potash, trona, salt, 
or limestone) (S2 estimation groups); and across 
all facilities (S3 estimation groups) because of the 
limited numbers of measurements. The assignments 
were made blinded to the measurement levels. Each 
job was assigned the AM of its surface exposure 
group (A-C) at the most specific facility level (Sl- 
S3) having at least five REC personal measurements. 

To evaluate the surface groupings, the AM of each 
exposure group's REC measurements and the 
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percentage of nondetectable (ND) measurements 
were calculated to determine if increasing REC 
levels, and decreasing percentages of ND measure- 
ments, were associated with increasing proximity 
to, or use of, diesel equipment. 

Only Pearson correlation coefficients are pre- 
sented in this report because the Spearman correla- 
tion coefficients were similar. All elements of the 
exposure assessment process, including this evalua- 
tion work, were done blind to the disease outcomes 
of the epidemiologic studies. 



RESULTS 

Jobs with missing location information 

For the 1640 jobs missing location data, there 
was 93% agreement (ranging from 86 to 100% by 
facility) between the location ascertained from the 
facility records and from the long-term workers. 
The agreement was similar for jobs identified in 
the records as being on the surface or as being under- 
ground (Supplementary data are available at Annals 
of Occupational Hygiene online). 

REC AMs as the exposure metric 

A sensitivity analysis compared the use of REC 
medians in the estimation of cumulative exposure 
levels of underground workers to cumulative levels 
based on the REC AMs that were used for the 
computation of cumulative exposure levels in the ep- 
idemiologic analysis. There was an excellent corre- 



lation between the two sets of cumulative exposure 
levels overall (r P = 0.98) and by facility (r P = 
0.98 to >0.99) (Supplementary data are available 
at Annals of Occupational Hygiene online). 

Underground exposure estimates 

Development of underground exposure groups. A 
TWA was derived from the 1998 to 2001 REC area 
measurements and estimates of the proportion of 
time spent in the four underground areas for each 
job with 1998-2001 personal measurements and 
compared to the corresponding REC full-shift per- 
sonal AM that was used as the 1998-2001 reference 
value. The TWAs overestimated the full-shift meas- 
urements by -19% (-48 to 20% by facility) 
(Table 2). The median correlation was 0.83 (0.15- 
0.72 by facility). 

The analysis of variance results on the under- 
ground job groups' REC measurements indicated 
that the Ul set of groups (i.e. standardized job titles) 
explained more between-group variability than the 
U2 and U3 sets of groups in three facilities (Facili- 
ties B, E, and G), but they explained little of the 
variance for the other facilities (Table 3). There 
was little difference in the within-job or -group var- 
iance across the three sets of groups for any facility. 
For example, Facility B's between-group variability 
was 0. 12 for its Ul groups and 0.00 for its U2 and U3 
groups, whereas the within-group variability was 
essentially the same for the three sets of groups 
(i.e. 3.28, 3.24, and 3.27, respectively). No change 
to the exposure group assignments was made as 
a result of this evaluation. 



Table 2. Comparison of estimated REC 8-h TWA a personal levels to DEMS REC full-shift personal measurements for 
underground jobs. 



Facility b 


WJobs 


AM of the DEMS REC personal 
measurement AMs (ug m~ 3 ) 


AM of estimated 
REC TWAs (ug nT 3 ) 


Difference 
(ug m~ 3 ) 


Relative difference" 1 
(%) 


rp 


A 


14 


339 


472 


-133 


-39 


0.39 


B 


9 


107 


158 


-52 


-48 


0.52 


D 


14 


109 


88 


22 


20 


0.15 


E 


17 


65 


88 


-23 


-35 


0.72 


G 


11 


38 


38 


0 


1 


0.68 


H 


11 


84 


108 


-24 


-29 


0.62 


I 


21 


63 


69 


-7 


-11 


0.21 


All 


97 


113 


135 


-22 


-19 


0.83 



N, number of jobs; r B Pearson correlation coefficient. 

""Estimated REC TWA (in micrograms per cubic metre) was calculated, by facility and overall as: [area concentration face x (% 
time face / 100)] + [area concentration halllage/travel ways x (% time hau | age/ttavl5l ways /100)] + (area concentration shop & office x (% 
time shop & offic e /100) + [area concentration,.,™,,,,,, x (% time crusheI /100)]. 

b Measurements in Facility J were not available because the facility had closed prior to the monitoring. 

c Relative difference was calculated by dividing the difference by the corresponding observed AM. The differences and relative 
differences may not be exact because of rounding. 
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Table 3. Variance components for log-transformed DEMS REC personal measurements of underground exposure groups, by 
facility. 



Facility" 


Ul exposure groups' 1 




U2 


exposure groups 




U3 exposure groups 




N 


S 2 


S 2 c 

1 W 1 


N 


S 2 

BG 


s 2 

WG 


N 


S 2 

°BG 


V 2 

WG 


A 


13 


<0.01 


0.24 


5 


0.03 


0.24 


3 


<0.01 


0.24 


B 


9 


0.12 


3.28 


5 


0.00 


3.24 


4 


0.00 


3.27 


D 


8 


0.16 


0.54 


5 


0.35 


0.46 


3 


0.54 


0.46 


E 


16 


0.48 


0.81 


7 


0.15 


0.80 


3 


0.16 


0.73 


G 


13 


0.35 


0.72 


5 


0.10 


0.70 


3 


0.15 


0.69 


H 


16 


0.00 


1.21 


6 


0.00 


1.23 


3 


0.00 


1.22 


I 


21 


0.08 


0.73 


6 


0.07 


0.75 


3 


0.07 


0.73 


N, number of groups 


in each facility; S 2 ^ 


, between-job variance; S 2 ^, within-job 


variance; 


Sg G , between- 


group variance 


■ S 2 

' J WG' 



within-group variance. 

"Measurements in facility J were not taken because the facility had closed prior to the monitoring. 

b Ul exposure groups were standardized job titles of underground jobs and covered 40% of the person-years of the study. U2 
exposure groups comprised sets of standardized job titles based on the percent of time workers with these jobs worked in each of 
four underground areas (the production face, haulage or travel ways, maintenance shop and offices and, where existent, crusher 
area) (40% of the person-years). U3 exposure groups comprised sets of U2 groups, based on similar CO area air concentrations 
(6% of the person-years). The two remaining grouping methods, representing 14% of the person-years, could not be evaluated. 
Includes within-day variability. 



Assumptions used for modeling historical trends. 
There was little difference between the primary 
REC estimates for underground jobs in 1976 and 
the REC estimates based on the changing relation- 
ship of DPM and CO in engine emissions from the 
1990s back to 1976, as published by Yanowitz 
et al. (2000) and the US EPA (2002). The 1976 
REC estimates would have been only 10% lower 
than estimated in the study had the Yanowitz et al. 
(2000) data represented the trends in engine emis- 
sions in our facilities. 

The sensitivity analysis exploring the robustness 
of the assumptions used in the time trend prediction 
models generally found very high correlations be- 
tween the cumulative exposure levels of under- 
ground workers using alternate assumptions with 
those derived from the primary models. The results 
from the model using the REC = CO 0 ' 58 relationship 
led to an overall correlation of 0.88 (r P = 0.96-0.99 
by facility) (Supplementary data are available at An- 
nals of Occupational Hygiene online). The 5-year 
average CO estimates (i.e. using only the measure- 
ment data after 1975 and the ADJ/CFM ratio before 
that) resulted in an overall correlation with the 
primary estimates of 0.87 (r P = 0.95 to >0.99 by 
facility). 

Time trends. Comparison between observed un- 
derground 1976-1977 CO face measurement AMs 
and the underground 1976-1977 CO face concentra- 
tions estimated from the primary prediction models 
found that the median of the relative differences be- 
tween the two sets of estimates was 29% (—25 to 
49% by facility) (Table 4). Figure 1 displays the 



1976-1977 CO estimates and the variability of the 
1976-1977 CO measurements. Comparison of the 
1994 REC estimates developed from the primary 
models to 1994 REC measurements in Facility B 
found that the primary estimate was higher than 
the measurement mean for the continuous miner by 
10% and for the foreman by 6%. 

Surface exposure estimates 

Overall, the estimated REC levels for Surface ex- 
posure groups A, B, and C increased with increasing 
contact with DE (1, 3, and 5 ug m~ 3 , respectively) 
(Table 5). The AMs also increased monotonically 
across the exposure groups in five facilities (Facili- 
ties B, D, H, I, and J). In Facility A, the estimate 
for exposure group A was overridden due to a rela- 
tively higher concentration when compared with 
the estimate for Exposure group B in that facility 
or for Exposure groups A or B in any other facility. 
For all facilities combined, the percentage of NDs 
decreased as surface exposure groups potentially 
had more contact with DE (75, 57, and 47% respec- 
tively, for Exposure groups A, B, and C). 



DISCUSSION 

Evaluating exposure assessment methods is an im- 
portant component of epidemiologic studies. Stud- 
ies, however, often lack the data to make such 
evaluations. In this study, we collected information 
that could be used to evaluate the exposure assess- 
ment process because we had no gold standard. We 
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Table 4. Comparison of underground estimated CO production face area and REC personal levels to independent CO production 
face area and REC personal measurement AMs, by facility and overall. 





Measurements 
N 


AM 


Estimates 


Difference 


Relative 
difference" (%) 


1976-1977 CO area (p.p.m.) b 












Facility 0 












B 


90 


7.2 


5.2 


2.1 


29 


D 


136 


10.5 


8.0 


2.5 


24 


E 


148 


8.5 


10.6 


-2.1 


-25 


H 


100 


7.7 


3.9 


3.8 


49 


I 


122 


7.7 


4.9 


2.9 


37 


J 


217 


8.1 


4.4 


3.7 


46 


All 


813 


8.3 


6.1 


2.2 


29" 


1994 REC personal (ug m~ 3 ) e 












Job title 












Continuous miner 


26 


248 


273 


25 


-10 


Foreman 


6 


166 


176 


10 


-6 



N, number of measurements; p. p.m., parts per million. 

"Relative difference was calculated by dividing the difference by the corresponding observed AM. Differences or relative 
differences may not be exact because of rounding. 
b From Sutton et al. (1979). 

c 1976- 1977 measurement data were not available for facilities A and G. 
d Median of the differences. 
e From Stanevich et al. (1997). 



O 
o 



Facilities 

Rel.Diff 

N 



B 

-29% 
90 



D 

-24% 
136 



E 

25% 
148 



H 

-49% 
100 



I 

-37% 
122 




J 

-46% 
217 



Fig. 1. CO, Carbon monoxide; Rel. Diff, Relative difference. For the definition, see footnote a in Table 4; N, Number of CO 
measurements from 1976-1977 MESA/BoM survey Sutton et al., (1979). The boxes display the 25th and 75th percentiles, the 
horizontal line within each box displays the median, and the number plotted represents the mean of the MESA/BoM survey. The 
vertical whiskers extend 1.5 times the interquartile range above and below the boxes. The dots indicate the predicted 1976-1977 

CO estimates by mining facility. 
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Table 5. Estimated REC personal exposure levels and percentage of nondetectable levels from the DEMS REC personal 
measurement data, by facility and surface exposure category. 



Facility 


Surface exposure group 


A a 


Surface exposure group B 


Surface exposure group C 




Estimated 
tsjz\~ level 
(ug m- 3 ) (TV) 


% ND 


Estimated 
KEL. level 
(ug nT 3 ) (TV) 


% ND 


Estimated 
JvbL, level 
(ug nT 3 ) (TV) 


% ND 


A 


2 C (96) 


75 


4 (11) 


9 


11 (6) 


0 


B 


1 (9) 


56 


3 (50) 


34 


5 d (34) 


47 


D 


1 (20) 


95 


2(12) 


83 


5 d (34) 


47 


E 


1 (6) 


100 


4(11) 


64 


4(8) 


75 


G 


2(12) 


75 


2(17) 


94 


4 e (17) 


53 


H 


1 (8) 


88 


2 (22) 


77 


4 e (17) 


53 


I 


1 (25) 


92 


2(10) 


80 


5 (12) 


42 


J f 


1 (29) 


83 


3 (62) 


44 


5 d (34) 


47 


All 8 


1 


75 


3 


57 


5 


47 



N, number of measurements; %ND, percent of nondetectable measurements. 

"Surface exposure category definitions — A: jobs in which workers had no or very limited contact with diesel equipment; B: jobs 
in which workers drove a diesel forklift truck indoors or operated heavy equipment (>75 HP) <4 h per shift on average, drove 
light diesel equipment (<75 HP), or worked in close proximity to diesel-powered equipment on a regular basis; and C: jobs in 
which workers operated heavy diesel equipment or drove a diesel forklift truck indoors for >4 h per shift on average, and 
mechanics and maintenance workers who repaired diesel equipment. 

b For the calculation of the AM, the concentrations for the ND measurements were imputed based on a distributional Maximum 
Likelihood Estimation procedure (Helsel, 2005; Vermeulen et al., 2010a). 

c The AM of the measurements (AM = 5 ug m~ 3 ; n = 6) for this exposure category was higher than the AM for Surface category 
B in this facility (AM = 4 ug m~ 3 ; n = 11) and two to five times higher than the same surface category in any other facility. 
Measurements for this category were derived from the AM of Surface category A measurements, pooled across all facilities. 
d The AM was derived from the AM of Surface category C measurements pooled across all facilities because there were fewer 
than five measurements for this category in each facility and in all potash facilities combined. 

e The AM was derived from the AM of Surface category C measurements pooled across facilities of the same ore type (i.e. trona) 
(G, H, and I) because there were fewer than five measurements for this category in Facilities G and H. 
f The AMs for Facility J were derived by pooling the measurements for Facilities B and D for Exposure category A and B. 
E The number of measurements on which the estimated AMs were based does not add up to the total number of measurements 
because of the grouping strategy when there were fewer than five measurements per facility/surface category. See text for details. 



compared DE-related historical exposure estimates 
to DE-related measurements, i.e. 1976-1977 CO 
measurement data and 1994 REC measurement data. 
We compared our estimates to recent data (1998— 
2001 REC area measurements to 1998-2001 REC 
personal measurements and trends in 1998-2001 
REC personal measurements by surface category). 
We also conducted sensitivity analyses to evaluate 
our assumptions. Despite the inherent limitations in 
the data and the comparisons, we found moderate 
to excellent agreement between our estimates and 
these other data sources. 

A critical evaluation is the comparison of our 
1976-1977 CO estimates predicted for the face to 
the CO measurement data collected in 1976-1977 
at the face (Sutton et al., 1979). We found little over- 
all difference between the two sets of data: the me- 
dian of the differences between the estimates and 
the observed measurements was 29% and ranged 
from —25 to 49% by facility (Vermeulen et al, 
2010a). In addition to the CO comparisons, two un- 
derground jobs with 1994 REC estimates could be 



directly matched to the measured REC in 1994 in Fa- 
cility B (Stanevich et al, 1997). The REC estimates 
were higher than the REC measurements by <10%. 

When we compared these differences to measure- 
ment error reported in the literature, we found simi- 
lar levels of agreement. In a study of side-by-side 
acrylonitrile measurements, in which two sets of 
measurements were taken concurrently on the same 
individuals using the same sampling and analytic 
method, mean differences of —14 to 21% were 
found, depending on the sampling method (Zey 
et al., 2002). A study of elemental carbon measure- 
ments taken side-by-side with our measurements 
generally found mean differences of —18 to 40% 
by facility and job function (Cohen et al., 2002) 
(Supplementary data are available at Annals of Oc- 
cupational Hygiene online). Thus, the relative differ- 
ences between the estimates and the measurement 
data were in the range of variability that has been 
seen in actual measurement studies. These results in- 
dicate that the estimates were reasonably accurate 
and support the use of the MSHA area CO 
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compliance measurement data, estimates of ADJ HP 
and other determinants to predict relative trends in 
DE exposure levels. 

Grouping of jobs is common in epidemiologic 
studies because measurement data are often sparse. 
As has been done by others (Quinn et al, 2001), 
we used job elements (which also could be called ex- 
posure determinants, e.g. percent time of a work shift 
spent in various underground areas) based on infor- 
mation obtained by interview to group jobs with 
and without DEMS REC measurements. We asked 
the workers to identify historical changes to ensure 
that our data reflected the standard procedures fol- 
lowed over time. Only a few jobs were reported as 
having changed and, in these cases, a new job title 
was developed to allow a different set of estimates. 
When we compared estimated TWAs, calculated 
from the DEMS area measurements and proportion 
of time in these areas, with full-shift DEMS personal 
measurements, the relative difference was —19%. 
This difference is similar to that found for side- 
by-side measurements (Cohen et al., 2002; Zey 
et ah, 2002). The overall correlation between the 
two sets of data was 0.83. The two facilities with 
the low correlations (A and I) were likely due to 
the small number of measurements (<5) for several 
of the jobs and areas used in the analysis and the ho- 
mogeneity of exposure levels in these facilities. 
These findings are important because the basis for 
assignment of a substantial number of jobs to expo- 
sure groups was the proportions of time spent in the 
various underground areas. The results suggest that 
the time estimates were accurate enough to rank 
the jobs similarly to the measurement data. 

The goal of any grouping strategy is to maximize 
between-job (or group) variability and minimize 
within-job (or group) variability. Grouping of jobs 
is, therefore, a trade-off between reliability and spec- 
ificity. We chose to use facility-specific standardized 
job titles as the preferred method over cruder ways of 
grouping of underground jobs (e.g. departments, fa- 
cility, type of product) to increase specificity but 
used a criterion of at least five measurements per ex- 
posure group in order to increase the reliability of the 
means. The analysis of variance on the DEMS sur- 
vey data indicated that the selection of this approach 
over the two broader groups was appropriate in three 
facilities; in three others, it did not make a difference; 
and in only one facility was either of the other two 
types of groups preferable. This facility, however, 
had the lowest percentage of exposure-years linked 
to Ul groups (Stewart et al., 2010). The lack of con- 
trast among jobs in some of the mines was not so 
much a limitation of our job grouping strategy as it 



was a reflection of the homogeneity of exposure lev- 
els across jobs within the underground operations. 
Thus, these data support our use of standardized 
job titles as the preferred grouping strategy. Using 
the broader groups to represent jobs of possibly dif- 
ferent exposure levels was likely to have increased 
random error in the assigned exposure levels and 
to have decreased between-job variability, but some 
type of broader group was necessary for the jobs 
with no measurements. Thus, some attenuation of 
the lung cancer risk estimates may have occurred be- 
cause of the grouping procedure. 

The findings from the analyses of variance, i.e. 
that there were differences in underground jobs by 
facility as well as the findings of the differences 
among the facilities seen both with the personal 
measurements (Coble et al., 2010) and with the area 
measurements (Coble et ah, 2010; Vermeulen et al., 
2010b), support our development of facility-specific 
estimates. We applied the same procedure to all 
study facilities for consistency and because of the 
substantial differences in horsepower, ventilation 
levels, and other environmental factors between the 
underground operations (Coble et al., 2010; Stewart 
et al, 2010). 

The overall correlation of the cumulative exposure 
levels of underground workers using the medians 
with those using AMs was 0.98, with similar findings 
for the individual facilities. In part, these high corre- 
lations are due to the use of the same duration (i.e. 
number of years worked at a job) values used in both 
calculations of cumulative exposure. Nevertheless, 
these correlations suggest that using the AMs, with 
the occasional inclusion of extreme values, yielded 
almost the same ranking of study subjects as using 
medians, which are less affected by extreme values. 

Two sets of cumulative exposure estimates were 
developed from alternate prediction models to eval- 
uate the robustness of the study assumptions used to 
develop the primary historical REC estimates for un- 
derground jobs. The correlation coefficient between 
the subjects' cumulative REC estimates using the 
primary historical estimates and those using the 
power relationship of REC = CO 058 was 0.88. 
The coefficient comparing cumulative estimates 
based on the primary estimates with those based only 
on CO measurement data after 1975 was 0.87. Al- 
though the similarity of the correlations was influ- 
enced by the use of the same duration values in the 
calculation of cumulative exposure, these results 
suggest that the rankings of the study subjects by cu- 
mulative exposure were probably not significantly 
affected by the deterministic assumptions (e.g. 
ADJ HP/CFM) or the proportionality assumption 
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(change in COxhange in REC = 1:1) on which the 
primary models were based. 

The surface REC estimates were based on the 
likely contact with diesel equipment within the var- 
ious jobs, and assignment of jobs to surface groups was 
made blinded to the measurement levels. We found an 
increasing trend in the estimated concentration relative 
to increasing proximity to diesel equipment for the 
measured jobs across all facilities and for several indi- 
vidual facilities. Discrepancies away from the ex- 
pected monotonic trends of REC levels based on 
diesel contact were generally small (<20%) and, in 
part, may have been due to small numbers of measure- 
ments. The impact on the epidemiologic study of 
these discrepancies is likely to be minimal because 
the absolute differences among the discrepant surface 
workers' exposure estimates was small (<5 |ig m~ 3 ) 
compared to the mean estimates of the underground 
workers (78-216 ug m~ 3 by facility). 

SUMMARY 

Several approaches were used to assess the DE ex- 
posure estimates developed for our epidemiologic 
study of underground miners. Direct comparison of 
historical CO face measurement data with estimated 
CO face concentrations from our facility-specific 
prediction models found excellent overall agreement 
that was generally within the limits of the sampling 
variation for measurements seen by others. Other 
evaluations of components of the exposure assess- 
ment process (identification of unknown location in 
the work histories, the use of arithmetic means to de- 
scribe exposure levels, development of exposure 
groups of underground jobs based on estimates of 
the amount of time workers spent at specific under- 
ground areas, the use of a different REC:CO propor- 
tionality due to changing engine technologies, the 
development of alternate estimates based on analy- 
ses of recent REC and CO measurements and based 
on historical CO measurements, and development of 
surface exposure groups based on contact with diesel 
equipment) indicated moderate to excellent levels of 
agreement. Thus, the overall evidence from these 
evaluations suggests that our REC estimates are 
likely to be accurate representations of historical 
personal exposure levels to diesel exhaust that are 
useful for an epidemiologic study. 

SUPPLEMENTARY DATA 

Supplementary data can be found at http://annhyg. 
oxfordjournals.org/. 
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